Neural Network Basic

Neural Network Example

Computer Vision

• Image Classification

Computer Vision

• Object Detection

Computer Vision

• Neural Style Transfer

Image Data

Convolutions Schematic

Edge Detection

Vertical Edge Detection

Parameters

Padding

• Pad so that output size is the same as the input size.

Strided convolutions

• \(n\times n\) (\(n=7\))
• \(f \times f\) (\(f=3\))
• no padding
• stride \(s=2\)

Summary of Convolutions

• \(n \times n\) image; \(f \times f\) filter
• padding \(p\); stride \(s\)
• Output: \(\left[ \frac{n+2p-f}{s}+1 \right] \times \left[ \frac{n+2p-f}{s} +1 \right]\)

Convolutions Over Volume

Your Turn: Number of Parameters in One Layer

Question: If you have 10 filters that are \(3 \times 3 \times 3\) in one layer of a neural network, how many parameters does that layer have?

Summary of Notation

If layer \(l\) is a convolution layer:

• \(f^{[l]}\) = filter size
• \(p^{[l]}\) = padding
• \(f^{[l]}\) = stride
• \(n^{[l]}_C\) = number of filters
• Each filter: \(f^{[l]} \times f^{[l]} \times n^{[l-1]}_C\)
• Activations: \(a^{[l]} \rightarrow n_H^{[l]} \times n_W^{[l]} \times n_C^{[l]}\)
• Weights: \(f^{[l]} \times f^{[l]} \times n^{[l-1]}_C \times n^{[l]}_C\)
• bias: \(n^{[l]}_C\)
• Input: \(n^{[l-1]}_H \times n^{[l-1]}_W \times n^{[l-1]}_C\)
• Output: \(n^{[l]}_H \times n^{[l]}_W \times n^{[l]}_C\)
• \(n^{[l]}_H = \frac{n^{[l-1]}_W + 2p^{[l]}-f^{[l]}}{s^{[l]}}+1\) (similar for \(W\))

Pooling Layers

Examples: LeNet - 5

LeCun et al., 1998. Gradient-based learning applied to document recognition

Examples: LeNet - 5

Types of Layer in A Convolutional Network

• Convolution

• Pooling

• Fully connected